Zooming control apparatus, image capturing apparatus and control methods thereof

ABSTRACT

A zooming control apparatus comprises an object detection unit configured to detect an object from an image; a first acquisition unit configured to acquire information regarding a distance to the object; and a zooming control unit configured to perform zooming control for automatically changing a zoom magnification according to at least one of second information that includes information regarding a size of the object detected by the object detection unit and first information regarding the distance to the object acquired by the first acquisition unit, wherein a condition for automatically changing the zoom magnification in the zooming control differs according to a reliability of the first information.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a divisional of application Ser. No. 16/233,625,filed Dec. 27, 2018, which is a continuation of application Ser. No.15/459,847, filed Mar. 15, 2017, which issued as U.S. Pat. No.10,200,620 on Feb. 5, 2019, the entire disclosures of which is herebyincorporated by reference.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an image capturing apparatus providedwith a zooming control apparatus.

Description of the Related Art

Some image capturing apparatuses such as digital cameras have a functionfor performing optical resizing (hereinafter, referred to as opticalzooming) by driving a zoom lens and a function for performing electronicresizing (hereinafter, referred to as electronic zooming) by enlarging aportion of an image. Furthermore, some image capturing apparatuses havean automatic zooming (automatic resizing) function for performing objectdetection and automatically changing the zoom magnification for aspecific object. Techniques for controlling a zoom operation using suchan optical zooming function and an electronic zooming function aredescribed in Japanese Patent Laid-Open No. 2015-43557 and JapanesePatent Laid-Open No. 7-135593.

Japanese Patent Laid-Open No. 2015-43557 describes an automatic zoomingfunction for preventing an object from being framed out and an automaticzooming function for maintaining the size of an object image constant.In Japanese Patent Laid-Open No. 2015-43557, the shape of the face of aperson or the characteristic color of an object other than a person isdetected, and in the case where the detected object moves in a directionoutside the angle of view and is about to be framed out of the angle ofview, frame-out is prevented by zooming out the zoom lens. Also, inJapanese Patent Laid-Open No. 2015-43557, in the case where the detectedobject moves in the front and back direction and thereby the size of theobject image changes, the zoom lens is driven so as to maintain theoriginal size.

Moreover, Japanese Patent Laid-Open No. 7-135593 describes the automaticzooming function for maintaining the size of the object imagesubstantially constant based on the focusing position of a focus lens.In Japanese Patent Laid-Open No. 7-135593, an object distance ismeasured from the focus position when the object is in focus withautofocus (AF) control, and it is detected that the object has movedrelative to the camera in the front and back direction, thereby changingthe object distance. The size of the object image is maintainedsubstantially constant by driving the zoom lens by a change amount ofthe size of the object image equivalent to the change amount of theobject distance.

However, in Japanese Patent Laid-Open No. 2015-43557 above, in the casewhere the characteristic color of an object other than a person isdetected as an object region, if a region of the same type of colorexists in the background or in an object other than a person, or if alight source changes, there is a possibility that a detection error ofthe position and size of the object region becomes large. Moreover,according to Japanese Patent Laid-Open No. 7-135593 above, in the casewhere the object distance is measured when focusing by AF is notcomplete, or in a region in which the depth of field is deep and theobject is focused over a wide distance with respect to a unique focusposition, there is a possibility that a measurement error of the objectdistance becomes large. If automatic zooming is performed on an objectother than a person in this manner, there is a possibility thatautomatic zooming will malfunction or not operate normally due to adetection error of the position and the size of the object image or ameasurement error of the object distance.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of theaforementioned problems, and realizes a technique that enablesappropriate zooming control according to a detected object.

In order to solve the aforementioned problems, the present inventionprovides a zooming control apparatus comprising: an object detectionunit configured to detect an object from an image; a first acquisitionunit configured to acquire first information regarding a distance to theobject; and a zooming control unit configured to perform zooming controlfor automatically changing a zoom magnification according to at leastone of second information that includes information regarding a size ofthe object detected by the object detection unit and the firstinformation regarding the distance to the object acquired by the firstacquisition unit, wherein a condition for automatically changing thezoom magnification in the zooming control differs according to areliability of the first information.

In order to solve the aforementioned problems, the present inventionprovides an image capturing apparatus comprising: the foregoing zoomingcontrol apparatus; an image capturing unit configured to capture anobject and generate image data; and a display unit configured to displaythe image, wherein the zooming control unit controls a zoommagnification of the object in the image displayed on the display unit.

In order to solve the aforementioned problems, the present inventionprovides a control method of a zooming control apparatus comprising:detecting an object from an image; acquiring first information regardinga distance to the object; and performing zooming control forautomatically changing a zoom magnification according to at least one ofsecond information that includes information regarding a size of thedetected object and first information regarding a distance to theobject, wherein a condition for automatically changing the zoommagnification in the zooming control differs according to a reliabilityof the first information.

In order to solve the aforementioned problems, the present inventionprovides a non-transitory computer-readable storage medium storing aprogram for causing a computer to execute a control method of a zoomingcontrol apparatus comprising: detecting an object from an image;acquiring first information regarding a distance to the object; andperforming zooming control for automatically changing a zoommagnification according to at least one of second information thatincludes information regarding a size of the detected object and firstinformation regarding a distance to the object, wherein a condition forautomatically changing the zoom magnification in the zooming controldiffers according to a reliability of the first information.

According to the present invention, appropriate zooming control can beperformed in accordance with a detected object.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an apparatus configuration ofembodiments of the present invention.

FIG. 2 is a diagram illustrating the relationship between a focal lengthand a focus lens position for each object distance.

FIG. 3 is a block diagram showing the configuration of an objectdetection unit.

FIG. 4 is a diagram showing the relationship between a focus positionand a contrast evaluation value.

FIG. 5 is a diagram showing state transition of a continuous AF mode.

FIGS. 6A and 6B are diagrams illustrating the relationship between anobject distance and a focus lens position for each focal length.

FIGS. 7A and 7B are diagrams illustrating processing for preventingframe-out of an object other than a person.

FIGS. 8A to 8C are diagrams illustrating processing for preventingframe-out of an object that is a person.

FIGS. 9A to 9F are diagrams illustrating processing for preventing sizechange of an object that is a person.

FIGS. 10A to 10F are diagrams illustrating processing for preventingsize change of an object other than a person.

FIG. 11 is a flowchart showing a processing procedure of automaticzooming control of a first embodiment.

FIGS. 12A to 12C are flowcharts showing detailed object designatingprocessing in FIG. 11.

FIG. 13 is a flowchart showing detailed reference distance settingprocessing in FIG. 11.

FIG. 14 is a flowchart showing detailed automatic zooming controlprocessing in FIG. 11.

FIG. 15 is a flowchart showing detailed frame-out prevention controlprocessing in FIG. 14.

FIG. 16 is a flowchart showing detailed size maintaining controlprocessing in FIG. 14 of the first embodiment.

FIGS. 17A and 17B are flowcharts showing detailed size maintainingcontrol processing in FIG. 14 of a second embodiment.

FIG. 18 is a flowchart showing a processing procedure during a zoomoperation of the embodiments of the present invention.

DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention will be described in detail below.The following embodiments are merely examples for practicing the presentinvention. The embodiments should be properly modified or changeddepending on various conditions and the structure of an apparatus towhich the present invention is applied. The present invention should notbe limited to the following embodiments. Also, parts of the embodimentsto be described later may be properly combined.

First Embodiment

A digital camera 100 that has an automatic zooming function will bedescribed below as an example of an image capturing apparatus providedwith a zooming control apparatus according to this embodiment.

Note that this embodiment is intended for an image capturing apparatussuch as a digital still camera and a digital video camera, but thepresent invention may be applied to a mobile phone equipped with acamera, or information processing apparatuses such as a smart phone,which is one type of such a mobile phone, and a tablet terminal.

Apparatus Configuration First, the configuration and the functions ofthe image capturing apparatus of this embodiment will be described withreference to FIG. 1.

A lens barrel 101 holds a lens group therein. A zoom lens 102 in thelens barrel 101 is moved in the optical axis direction so as to adjustthe focal length, and optically change the angle of view (move the zoomposition). Also, a focus lens 103 in the lens barrel 101 is moved in theoptical axis direction so as to perform focusing. A stabilization lens104 is a correction lens for correcting image blur caused by camerashake. An aperture-and-shutter 105 for adjusting a light amount is usedfor exposure control. Note that, in the digital camera 100 of thisembodiment, the lens barrel 101 and a camera body are integrallyconstituted, but the present invention is not limited thereto, and isalso applicable to an interchangeable lens type camera in which a lensunit is detachable from the camera main body, and the like.

An image sensor 106 generates imaging signals by receiving light thathas passed through the lens barrel 101, and converting the object imageinto electrical signals by photoelectric conversion. The image sensor106 is a CCD (charge coupled device) or CMOS (complementary metal oxidesemiconductor) image sensor, or the like. The imaging signals generatedby the image sensor 106 are input to an image processing unit 107,undergo various types of image processing such as pixel interpolationprocessing and color conversion processing, and are stored as image datain an image memory 108. The image memory 108 is constituted by a DRAM(Dynamic Random Access Memory), a SRAM (Static Random Access Memory) orthe like.

A display unit 109 is constituted by a TFT LCD (thin film transistorliquid crystal display) or the like, and displays shot image data andspecific information (e.g., shooting information and a tracking frame).An electronic viewfinder (EVF) function for the photographer to adjustthe angle of view is realized by such information display such as liveview of a shot image.

An aperture/shutter driving unit 110 calculates an exposure controlvalue (an aperture value and a shutter speed) based on luminanceinformation acquired by image processing in the image processing unit107, and drives the aperture-and-shutter 105 based on the calculationresult. Accordingly, automatic exposure (AE) control is performed. Astabilization lens driving unit 111 calculates a blurring amount appliedto the digital camera 100 based on information from an angular velocitysensor such as a gyro sensor, and drives the stabilization lens 104 soas to eliminate (or reduce) blurring.

A focus lens driving unit 112 drives the focus lens 103 based onautomatic focus adjustment (autofocus (AF)) control. For example, in thecase of performing contrast AF control, the focus lens driving unit 112drives the focus lens 103 based on focus adjustment information(contrast evaluation value) of the imaging optical system acquired bythe image processing unit 107 performing image processing, such that theobject is focused. Note that AF control in this embodiment is notlimited to contrast AF control, and a phase difference AF method may beapplied as another AF control, and AF control may be performed by aplurality of methods by combining a contrast method and another method.

The zoom lens driving unit 113 drives the zoom lens 102 in accordancewith a zoom operation instruction from the photographer. An operationunit 117 includes a zoom lever, a zoom button and the like as zoomingoperation members for the photographer to instruct the camera to performzooming. A system control unit 114 calculates a zoom driving speed and azoom driving direction based on an operation amount and an operationdirection of a zooming operation member, and performs control so as tomove the zoom lens 102 along the optical axis in accordance with thecalculation result.

Image data generated by performing a shooting operation is output to arecording unit 116 via an interface (I/F) unit 115 and is recorded. Theimage data is recorded in one of or both an external recording mediumand a non-volatile memory 118. The external recording medium is a memorycard or the like that is mounted to the digital camera 100 to be used.The non-volatile memory 118 is a storage medium build into the digitalcamera 100. The memory 118 stores, in addition to program data and imagedata, setting information of the digital camera 100 and information suchas a zoom-in position in an automatic zooming function, which will bedescribed later.

The operation unit 117 includes, in addition to the above zoomingoperation members, a release switch for instructing shooting start, anautomatic zooming switch for instructing the start and end of theautomatic zooming function, and the like. Operation signals from theoperation unit 117 are sent to the system control unit 114.

The system control unit 114 includes a calculation apparatus such as aCPU (central processing unit). The system control unit 114 performsoverall control of the camera by sending a control instruction to eachconstituent element in accordance with an operation of the photographer.Also, the system control unit 114 executes various control programsstored in the memory 118, for example, programs for performing controlof the image sensor 106, AE/AF control, zooming control (includingautomatic zooming processing) and the like.

Automatic Zooming Control Next, control related to the automatic zoomingfunction of the system control unit 114 will be described. Note thatFIG. 1 shows internal processing of the system control unit 114 usingfunction blocks 119 to 123.

If the lens barrel 101 is a rear focusing type, it is necessary to movethe focus lens 103 to an appropriate focus position in accordance withthe position of the zoom lens 102 in order to maintain an in-focus statewhen changing the angle of view by optical zooming. Such control iscalled computer zooming (CZ) control. This CZ control is performed by aCZ control unit 119 in the system control unit 114.

FIG. 2 shows, as a data table, a graph indicating the relationshipbetween a focal length of the zoom lens and a focus position at which anobject is focused, for each distance to the object. This table isreferred to as a focus cam table. In FIG. 2, the horizontal axisindicates a focal length (Focallength) corresponding to a zoom position,and the vertical axis indicates Focus Position, and a distance (objectdistance) from the camera to the object is shown next to each of theline graphs. The system control unit 114 controls the focus lens drivingunit 112 during an AF operation so as to move the focus lens 103 in apredetermined range, thereby performing a scan operation. A focusposition that is a focusing point is detected by a known method byacquiring a contrast evaluation value by this scan operation. Also, theobject distance can be acquired from the zoom position and the focusposition at that time by referring to the focus cam table.

Next, function blocks in the system control unit 114 related to theautomatic zooming function will be described. As shown in FIG. 1, thesystem control unit 114 is provided with the CZ control unit 119, anelectronic zooming control unit 120, a tracking frame control unit 121,an automatic zooming (AZ) control unit 122, an object detection unit 123and an AF control unit 124.

The digital camera 100 has an optical zooming function and an electroniczooming function. The CZ control unit 119 and the zoom lens driving unit113 handle driving control of the lenses related to the optical zoomingfunction. The CZ control unit 119 detects the zoom position of the zoomlens 102 for each predetermined control cycle during a zoom operation,and controls, in accordance with the detected zoom position, the zoomlens driving unit 113 so as to drive the focus lens 103 in conformitywith the focus cam table for the object distance acquired by an AFcontrol unit 124, which will be described later. This makes it possibleto perform an optical zoom operation while maintaining the in-focusstate.

On the other hand, the electronic zooming control unit 120 and the imagememory 108 handle control related to the electronic zooming function.The electronic zooming control unit 120 realizes the electronic zoomingfunction by segmenting data of a target region from image datatransferred to the image memory 108. The electronic zooming control unit120 then realizes smooth electronic zooming display by causing thedisplay unit 109 to display the data while gradually increasing therange that is segmented in a frame rate cycle of the image that isloaded into the image sensor 106.

The object detection unit 123 detects a desired object region from animage region of image data stored in the image memory 108. Here, facedetection processing and color detection processing as object detectionmethods for detecting an object (the face of a person or the like and abody other than a person) based on face information of the person orcolor information included in the image data will be described. FIG. 3shows the configuration of the object detection unit 123 that performsobject detection processing of this embodiment. The object detectionunit 123 includes a characteristic region detection unit 300, a trackingunit 301 and a reliability calculation unit 302. The characteristicregion detection unit 300 detects, from an image, the characteristicamount of a face in the case where the object is the face of a person,and detects, from the image, the characteristic color of a body in thecase where the object is a body other than a face of a person, in orderto estimate an object region (the position and the size of the object inthe image). The tracking unit 301 determines an object that moves over aplurality of consecutive frame images, and tracks the same objectregion. The reliability calculation unit 302 calculates the probabilitythat the object region detected by the characteristic region detectionunit 300 is the object, as the reliability (likelihood) of the object.

A frame control unit 121 displays, on the display unit 109, an objectdesignating frame serving as a guide for the photographer to designatean object. The frame control unit 121 also displays, on the display unit109, an object tracking frame so as to surround the object image suchthat the object designated by the photographer can be visuallyrecognized. The position and the size of the frame on the screen arecalculated by the tracking unit 301 based on face information and colorinformation, and is updated at a frame rate cycle.

First, face detection processing and color detection processing of thecharacteristic region detection unit 300 will be described.

The face detection processing is processing for detecting a face regionthat exists in image data using a known algorithm. For example, thecharacteristic region detection unit 300 extracts a characteristicamount from a square partial region in the image data, and compares thecharacteristic amount to a face characteristic amount prepared inadvance. If the correlation value of those amounts exceeds apredetermined threshold, the characteristic region detection unit 300determines that the partial region is a face region. Various faceregions that exist in the image can be detected by repeating thisdetermination processing while changing the combination of the size, thearrangement position and the arrangement angle of the partial region. Inaddition, in the case where there is a face authentication function,pattern matching between characteristic amounts of face imagesregistered in advance and a characteristic amount of a detected faceregion is executed by a known algorithm, and a registered face imagewhose correlation value is the highest is authenticated as the detectedface. Moreover, in the case where the correlation values for all of theregistered face images are smaller than the predetermined value, it isdetermined that the detected face region is a face that is notregistered.

In the color detection processing, the color information of an objectregion designated in accordance with an object designating method, whichwill be described later, is stored as a characteristic color. The colordetection processing is executed in the case where a detected object isa body other than a face of a person. RGB signals that are outputsignals from the image processing unit 107, luminance signals (Y), colordifference signals (R-Y, B-Y) and the like are used as the colorinformation. During object detection, the characteristic regiondetection unit 300 divides image data into a plurality of partialregions, and calculates the average value of luminance and colordifference for each of the partial regions. The characteristic regiondetection unit 300 also compares characteristic color information storedin advance to the color information of each region during the objectdetection, and sets, as candidates for an object region, partial regionsin which the differences in luminance and color difference are smallerthan or equal to predetermined amounts. A group of adjacent partialregions that are the candidates for an object region is set as a regionof the same color, and the region of the same color that is in apredetermined size range is set as a final object region. If the objectis a body other than a face of a person, there is a possibility that theshape of the object region is different depending on the body. In viewof this, a minimum square region that includes all of the regions of thesame color is set as a body region. Therefore, the body region may be arectangular region whose sizes in the horizontal direction and thevertical direction are different. In addition, the central position ofthe square region is used as the position of the body region.

Next, object tracking processing by the tracking unit 301 will bedescribed. The object tracking processing is processing for continuouslytracking an object region that moves over consecutive frame images as afocusing target. When the automatic zooming function is started, theregion of a person or a body serving as a main object is designated inaccordance with the object designating method, which will be describedlater. If the designated object is a person, namely, a face, all thefaces in the frame image are detected. If a plurality of faces aredetected in the same image, a face that is closest to the position ofthe face in the previously preceding frame image is determined as aperson to be tracked by the tracking unit 301. In addition, in the casewhere a detected face is a face registered by a face authenticationfunction, the authenticated face is preferentially determined as aperson to be tracked regardless of the positional relationship in frameimages. On the other hand, in the case where the designated object is abody other than a face of a person, processing for comparing thecharacteristic color information is started from the position of thebody region in the immediately preceding frame image. If the frame rateis sufficiently higher than the speed of the moving body, it is highlypossible that the same body will be detected in nearby regions inconsecutive frame images. Accordingly, the moving body region can betracked by the tracking unit 301 performing processing for matchingregions of the same color while shifting a partial region for thecomparison upward, downward, to the right and to the left, with animmediately preceding position of the body region serving as an origin.

Next, processing for calculating the reliability of an object performedby the reliability calculation unit 302 will be described. If a detectedobject is a face region, which is a region whose correlation value witha region constituted by face parts (e.g., a mouth, eyes and a nose) ishigh, deviation of the size of the detected face region from the size ofthe actual face is relatively small. In contrast, in the case where theobject is a body other than a face of a person, namely, in the case ofdetecting a region of the same color, there is a possibility thatdeviation of the position and the size of the body region is greatercompared to a face region, in the case where a region of the same typeof color exists in an object that is not the background or the mainobject, due to influence of a change of the light source, and the like.In view of this, in this embodiment, in the case where the object is abody other than a face of a person, automatic zooming is prevented frommalfunctioning by using object likelihood, namely, probability as anobject for determining whether or not to start a zoom operation. Factorsthat hinder object probability include change in the object, existenceof a similar object, and accumulation of tracking errors. The objectlikelihood is calculated by multiplying these causes by variousevaluation values acquired from matching processing and object regionestimation.

Greater difference between the luminance and the color difference of thecharacteristic color of an object stored when the object region isdesignated and the luminance and the color difference of a partialregion detected for each frame image indicates greater change of theobject. In view of this, the reliability calculation unit 302 calculatesa value obtained by normalizing, using the number of partial regions,the absolute sum of the difference in luminance and color differencebetween the characteristic color of the object and the partial regionsconstituting the body region, such that the greater this value is, thesmaller the likelihood of the object becomes.

In addition, in the case where a region whose color is the same as thecharacteristic color of the object is detected in a region other thanthe object region that is being tracked and has been determined by thetracking unit 301, it is highly possible that a similar object regionexists. If the position of the similar object region is close to theposition of the object region that is being tracked, there is apossibility that transfer has been made to another object. Moreover, thehistograms of the luminance and the color difference for each partialregion are calculated, and in the case of distribution concentrated on aspecific range, it is highly possible that the object and the backgroundhave the same type of color, or an object that is difficult to identifyhas been designated. In view of this, in the case where the distancebetween object regions is smaller than a predetermined value or in thecase where a specific histogram bin is greater than a predeterminedvalue, object likelihood is set to be smaller by the reliabilitycalculation unit 302.

Furthermore, once object probability becomes low, the reliability ofobject tracking after that also becomes low. Therefore, when calculatingthe likelihood of an object, the history of the likelihood of the objectis taken into consideration. For example, the average value of thelikelihood of the object for a predetermined period of time is handledas the likelihood of the object of the current frame image. Thereliability calculation unit 302 calculates object likelihood(reliability) in this manner.

When performing AF control, the AF control unit 124 controls the focuslens driving unit 112 so as to move the focus lens 103 in apredetermined range, thereby performing a scan operation. A contrastevaluation value acquired during the scan operation or the like is usedto detect a focus position that is a focusing point. An object distancecan be acquired with reference to the focus cam table using the zoomposition and the focus position at that time.

Autofocus Control Next, details of AF control performed by the AFcontrol unit 124 of this embodiment and an object distance acquisitionmethod will be described with reference to FIGS. 4 to 6B.

FIG. 4 illustrates the relationship between Focus Position and ContrastEvaluation Value in contrast AF control. The contrast evaluation valueis a value acquired by quantifying a high-frequency component of animage using a high pass filter or the like. A higher contrast evaluationvalue indicates a higher degree of focusing. In addition, AF control hastwo control modes, namely, a single AF mode that operates when therelease switch is half-pressed during still image shooting and acontinuous AF mode for keeping an object in focus in an EVF screen forframing the object at all times during moving image shooting. The singleAF mode is a mode for detecting, at a high speed, a focus position thatis the peak of the mountain of the contrast evaluation value, by movingthe focus lens 103 at one time so that the object distance changes frominfinity to close up range. In the single AF mode, although a focusingposition can be found at high speed, there are cases where the screenbecomes heavily blurred for an instant during a scan operation.Therefore, the single AF mode is suitable for AF control during stillimage shooting during which a release time lag is desired to beshortened as much as possible. On the other hand, the continuous AF modeis a mode for continuously focusing on an object by reversing the focuslens 103 in a range in which focus fluctuation is inconspicuous, forexample, in the case where a moving object or the like is continuouslytracked. Although it takes a long time to obtain an in-focus state froma heavily blurred state in the continuous AF mode, once an in-focusstate is entered, even if the object moves, it is possible tocontinuously focus on the object. Therefore, the continuous AF mode issuitable for AF control during moving image shooting in which the objectis continuously focused on at all times even in the case of an EVFscreen for framing an object and the case of a moving object. The AZcontrol unit 122 realizes the automatic zooming function by moving thezoom lens 102 based on a change amount of the object distance when theobject is focused in the continuous AF mode.

Next, focus following processing in the continuous AF mode will bedescribed with reference to FIG. 5. FIG. 5 illustrates state transitionof the continuous AF mode. States of the continuous AF mode includethree states, namely, a micro drive state 500, a hill-climbing drivestate 501 and a stopped state 502. When the continuous AF mode isstarted, transition is performed to the micro drive state 500 as aninitial state. In fine drive, a reverse operation of the focus lens 103is performed in the infinity direction and the close-range direction ina narrow range in which focus fluctuation is inconspicuous. If thecontrast evaluation value is small, in other words, in the case wherethe focus position is at the foot of the mountain of the evaluationvalue, the direction in which the evaluation value increases isdetermined by performing the reverse operation, and thereby it ispossible to determine whether the peak position of the mountain is inthe close-range direction or the infinite direction. When the directionof the peak of the mountain is determined in fine drive, transition isperformed to the hill-climbing drive state 501. In the hill-climbingdrive, the focus lens 103 is moved a large amount toward the directionof the peak of the mountain, and the peak position of the mountain atwhich the evaluation value starts to decrease after an increase isdetected. When the peak position is detected in the hill-climbing drive,transition is performed to the micro drive state 500 again. In finedrive, when the evaluation value is high, in other words, in the casewhere the focus position is near the peak position of the mountain ofthe evaluation value, the peak position of the mountain can be obtainedeven more minutely by determining the change in the evaluation valuewhile performing the reverse operation of the focus lens 103 more finelythan in the hill-climbing drive. If the focusing position is detected byfine drive near the peak position of the mountain, transition isperformed to the stopped state 502, where the focus lens 103 is stopped,and a focus flag indicating an in-focus state is set ON. In addition, ifa precise peak position could not be detected even by repeating thereverse operation in fine drive a predetermined number of times, thefocus lens 103 is moved again a large amount by the hill-climbing driveto detect the peak position. If the peak position of the mountain cannotbe detected even if the focus lens 103 is operated in the hill-climbingdrive to the infinity end or the close-up end, there is a possibilitythat the object is out of the range in which the object can be focused.Therefore, in this case, transition is performed to the stopped state502, where the focus lens 103 is stopped at the infinite end or theclose-up end, and the focus flag is set OFF. In the stopped state 502,it is periodically determined whether or not the evaluation value hasfluctuated. If the evaluation value fluctuates in the stopped state 502,it is determined that the object has moved, transition is performed tothe micro drive state 500 again, and the direction in which the objecthas moved is determined. In the continuous AF mode, focus cancontinuously follow a moving object by repeating fine drive,hill-climbing drive and stopping in this manner.

Next, a method for acquiring an object distance that is referred to inautomatic zooming control and the reliability of the acquired objectdistance will be described with reference to FIGS. 6A and 6B. FIG. 6Ashows the focus cam table shown in FIG. 2 graphed for each specificfocal length, as the relationship between the object distance (Distance)and the focus position (Focus Position) at which the object is focusedat the object distance. In FIG. 6A, the horizontal axis indicates theobject distance based on a logarithmic scale, and the vertical axisindicates the focus position. A numerical value shown beside each of theline graphs is a focal length corresponding to a zoom position. Thefocus cam table is a data group indicating focus positions correspondingto specific focal lengths and object distances, and is stored in thememory 118. The AF control unit 124 acquires a focus position from thefocus lens driving unit 112, and acquires a zoom position from the zoomlens driving unit 113 via the CZ control unit 119. An object distancecorresponding to the acquired focus position is searched for withreference to the focus cam table for a focal length corresponding to theacquired zoom position. If the same value as the acquired focus positionor zoom position is not included in the focus cam table, the objectdistance is calculated by performing interpolation calculation using aclose value included in the focus cam table. In addition, it becomespossible to more accurately obtain the object distance by obtaining, inadvance, a focus position at which an object at a known distance isfocused and adjusting data in the focus cam table individually.

FIG. 6B shows a graph of a focus position at which an object is focusedand that is differentiated using an object distance, namely, theinclination (the change amount of the focus position for an objectdistance of 1 m (hereinafter, referred to as a focus change rate)) ofthe graph in FIG. 6A. In FIG. 6B, the horizontal axis indicates anobject distance based on a logarithmic scale, and the vertical axisindicates a focus change rate relative to the object distance based on alogarithmic scale. A numerical value shown beside each of the linegraphs indicates a focal length corresponding to a zoom position. Asshown in FIG. 6B, the focus change rate increases as the focal lengthapproaches the telephoto side or the object distance approaches theclose range side, and decreases as the focal length approaches the wideangle side or the object distance approaches the infinity side. In arange in which the focus change rate is small, change in focus is smallrelative to the change in the object distance, and thus there is apossibility that the focus position is not changed by AF control even ifthe object actually moves. Therefore, it is conceived that the accuracyin obtaining the object distance, namely, reliability, is low in therange in which the focus change rate is small. In view of this, in thisembodiment, in the case where automatic zooming is performed inaccordance with the change in object distance, automatic zooming isprevented from malfunctioning by not executing automatic zooming that isbased on the object distance in a range in which the focus change rateis smaller than a predetermined value.

In addition, the focus change rate is a value that indicates a tendencysimilar to that of depth of field. Specifically, change in focus issmall in a range in which the focus change rate is small even if theobject distance changes, indicating that the depth of field is deep. Ina range in which the focus change rate is large, the focus changes evendue to a slight change in the object distance, indicating that the depthof field is shallow. In general, the depth of field has characteristicsof being shallower as the focal length approaches the telephoto side orthe object distance approaches the close range side, and being deeper asthe focal length approaches the wide angle side or the object distanceapproaches the infinity side, indicating a tendency similar to that ofthe focus change rate shown in FIG. 6B. In view of this, instead of thefocus change rate, the depth of field may be calculated as thereliability of the object distance, thereby determining whether or notto execute automatic zooming in accordance with the depth of field. Thedepth of field can be calculated by the following expressions.Front depth of field=(diameter of permissible circle ofconfusion×aperture value×object distance²)/(focal length²+diameter ofpermissible circle of confusion×aperture value×object distance)Rear depth of field=(diameter of permissible circle ofconfusion×aperture value×object distance²)/(focal length²−diameter ofpermissible circle of confusion×aperture value×object distance)Depth of field=front depth of field+rear depth of field

Note that the diameter of a permissible circle of confusion is a valuedetermined in accordance with the pixel size of the image sensor 106.Moreover, instead of calculating the depth of field, the focal length,the object distance and the aperture value that are terms constitutingthe expressions for calculating the depth of field may be individuallydetermined as the reliability of the object distance.

An attitude detection unit 125 detects the attitude of the camera (forexample, normal position/grip up/grip down) based on information of anacceleration sensor. A shake detection unit 126 detects a blurred stateof the camera based on angular speed information of the gyro sensor andthe like by performing determination. If a blurring amount (detectionvalue) applied to the gyro sensor or the like is greater than or equalto a predetermined amount (threshold), the shake detection unit 126determines that the camera is held by a hand, and in the case where theblurring amount is smaller than the predetermined amount, determinesthat the camera is fixed to a tripod or the like. A configuration may beadopted in which the acceleration sensor and the gyro sensorrespectively used for attitude detection and shake detection are usedalong with a detection sensor for acquiring control information of thestabilization lens driving unit 111.

Next, an overview of the automatic zooming function in this embodimentand the AZ control unit 122 will be described. In a camera that does nothave the automatic zooming function, in cases such as those where theobject moves and is framed out while the photographer is waiting for aphoto opportunity after performing framing in a telephoto state, thefollowing operation was necessary.

First, the photographer performs a zooming out operation using the zoomlever or the like included in the operation unit 117, and then searchesfor an object. After searching for an object, the photographer performsa zoom operation again until the desired angle of view is acquired, andadjusts the angle of view. Also, in cases such as where the object hasmoved and thus the size of the object image has changed, thephotographer needs to adjust the size of the object image by operatingthe zoom lever or the like.

On the other hand, in a camera that has an automatic zooming function,it suffices for the photographer to set the automatic zooming function,and then designate an object desired to be shot by performing anoperation of designating the object using a touch panel or the like.When the automatic zooming function is set, a zoom operation isautomatically performed such that the designated object fits apredetermined size near the center of the image. Note that methods fordesignating an object includes, in addition to a touch panel operation,a method for designating an object near the center of the screen whenthe photographer operates a specific button, and a method forautomatically selecting a main object from objects detected by thecamera.

The object detection unit 123 calculates the position and the size ofthe designated object region on the image data from the image memory108. It becomes possible to track the movement of the object byconsecutively performing this processing on sampling image data everytime the image is displayed as a live view. If the image of the objectthat is being tracked is detected in a zoom-out region, which will bedescribed later, and in the case where the detected object image islarger than a predetermined size, the AZ control unit 122 starts azoom-out operation. Specifically, the AZ control unit 122 instructs theCZ control unit 119 or the electronic zooming control unit 120 toperform zooming out in the wide angle direction. If an object isdetected in a zoom-in region, and the object image fits within a rangeof a predetermined size, a zoom-in operation is performed to thetelephoto side. With such processing, it suffices for the photographeronly to move the camera so as to fit the image of a desired objectwithin the screen without considering a zoom operation. Even in the casewhere the object is about to be framed out, the zoom magnification isautomatically changed, and thus the angle of view can be adjusted moreeasily.

Conditions for Determining to Start Zoom Operation

Next, conditions for determining to start a zoom-out operation and azoom-in operation will be described with reference to FIGS. 7A to 10F.FIGS. 7A and 7B show processing for preventing an object other than aperson from being framed out of the screen. FIGS. 8A to 8C showprocessing for preventing a person from being framed out of the screen.

In FIGS. 7A to 8C, a frame 700 a and a frame 700 b are first trackingframes (hereinafter, referred to as entity tracking frames) for trackingan object other than a person (entity), and frames 800 a to 800 f aresecond tracking frames (hereinafter, referred to as face trackingframes) for tracking an object (a face of a person). Hereinafter, in thecase where an object is applicable to both a face of a person and a bodyother than a face of a person, an entity tracking frame and a facetracking frame may be referred to as object tracking framescollectively. An object tracking frame is displayed so as to surround anobject image on the EVF screen of the display unit 109, such that theobject designated by the photographer can be distinguished. The positionand the size of the object tracking frame on the screen is calculated bythe tracking unit 301 of the object detection unit 123 based on faceinformation and color information, and is updated at the frame ratecycle.

Processing for preventing an object (airplane) from being framed out ofthe screen will be described with reference to FIGS. 7A and 7B. In FIG.7A, the zoom-out (ZO) region indicates a region that is outside theentire angle of view (the entire screen) displayed on the EVF screen, bymore than a predetermined ratio. For example, the case where the centralpoint of the screen is set as 0%, the entire screen is set as 100%, andpositions at which 80% of the entire screen is defined form a boundaryof the ZO region is envisioned. In this case, a region that is 80 to100% of the entire screen serves as the ZO region. When a portion of theentity tracking frame 700 a enters the ZO region, the AZ control unit122 performs control so as to start a zoom-out operation. The AZ controlunit 122 also stores, in the memory 118, the zoom magnification beforethe zoom movement (equivalent to a zoom-in angle of view). The targetzoom magnification and zooming speed during the zoom-out operation areset in advance in accordance with the size and the moving speed of theobject image. In addition, the target zoom magnification and zoomingspeed may be calculated in accordance with the size and the moving speedof the object image as appropriate. A zoom-out operation is executed inaccordance with the target zoom magnification and the zooming speed.Accordingly, frame-out of the object can be effectively prevented.

FIG. 7B shows an angle of view when a zoom-out operation equivalent to apredetermined zoom change amount is performed from the angle of view inFIG. 7A. The zoom-in (ZI) region shown in FIG. 7B indicates a regionwithin a zoom-in angle of view (angle of view before zooming out) 701 bymore than the predetermined ratio in an object searching state. Forexample, the case where the central point of the screen is set as 0%,the zoom-in angle of view 701 is set as 100%, and positions at which 70%of the zoom-in angle of view 701 is defined form a boundary of the ZIregion is envisioned. In this case, a region that is 0 to 70% of theentire zoom-in angle of view 701 serves as the ZI region. At this time,for example, if the zoom-out magnification is ½, the size of the zoom-inangle of view 701 is 50% of the entire screen. Therefore, it can be saidthat the boundary of the ZI region is 70%×(½)=35% of the entire screen,and the ZI region is a region that is 0 to 35% of the entire screen.When the photographer changes the attitude of the camera such that theentity tracking frame 700 b fits within the ZI region, the AZ controlunit 122 performs control so as to start a zoom-in operation.

Next, frame-out prevention control for preventing an object (person)from being framed out of the screen in a zoom operation of thisembodiment will be described with reference to FIGS. 8A to 8C. If theobject is a face of a person, when a portion of the face tracking frameenters the ZO region, a zoom-out operation starts, and when the facetracking frame fits within the ZI region, a zoom-in operation isperformed. If the object is a face of a person, unlike the case of abody other than a face of a person, the direction of movement of theobject can be predicted to some extent, and thus the ZO region and theZI region are set in accordance with a region in the direction of themovement that is predicted. In addition, in handheld shooting, theobject may be framed out due to the influence of camera shake and thelike. However, in the case where the object is framed out due to camerashake or the like, the object can be framed in again by the photographerperforming an operation of attempting to frame in the object. Here, inthe case where the ZO region is set in an upper portion of the screen,even when a shot is taken with a person arranged near the center, theface tracking frame may enter the ZO region, unintentionally causingzooming out. In view of this, in the case where the object is a face ofa person and the camera is handheld (in the state where the photographeris holding the camera), the ZO region is not set in an upper portion ofthe screen in consideration of a framing operation by the photographer.

In this manner, in this embodiment, in the case where a face is detectedby the object detection unit 123, the AZ control unit 122 changes the ZOregion and the ZI region in accordance with the attitude and the shakestate of the camera. The attitude (e.g., a vertical position and ahorizontal (normal) position) of the camera is detected by the attitudedetection unit 125, and in addition, the shake state of the camera isdetermined from a result of detection by the shake detection unit 126.The result of detection by the shake detection unit 126 is informationindicating whether or not the camera is held by a hand. Processing forsetting a zoom-out region and a zoom-in region according to thisembodiment will be described specifically below with reference to FIGS.8A to 8C.

FIG. 8A shows ZO regions and a ZI region that are set when thephotographer holds the camera by hand in a normal position. In such ashooting scene, in the case where the object moves in the horizontaldirection and is framed out, the position of the object in the screenmoves in the horizontal direction (longer side direction) for the screenin the normal position. Accordingly, both the ZO regions and the ZIregion are arranged in a form of a longitudinal band in the verticaldirection (short-side direction) for the screen at the normal position.The ZO regions are set in a form of a longitudinal band positioned atthe two ends of the laterally long rectangular screen in the horizontaldirection, and the ZI region is set in a form of a longitudinal bandpositioned at a center portion of the screen. In this case, when theface tracking frame 800 a enters the ZO region, the AZ control unit 122determines to start zooming out, and performs a zoom-out operationcorresponding to a predetermined zoom magnification. In addition, whenthe face tracking frame 800 b is included in the ZI region, the AZcontrol unit 122 determines to start zooming in, and performs a zoom-inoperation corresponding to the predetermined zoom magnification to azoom return position. Object (person) frame-out can be effectivelyprevented by setting the ZO regions and the ZI region in this manner.

FIG. 8B shows ZO regions and a ZI region that are set when the cameraattitude is changed in a shooting scene similar to that in FIG. 8A, andthe photographer holds the camera in a vertical position state, namely,grip up or grip down. In this case, the ZO regions and the ZI region arearranged in a form of a longitudinal band in the vertical direction(longer side direction) for the screen in the vertical position.Specifically, the ZO regions are set in a form of a longitudinal bandpositioned at the two ends of the vertically-long rectangular screen inthe horizontal direction, and the ZI region is set in a form of alongitudinal band positioned in the center portion of the screen. Inthis case, when the face tracking frame 800 c enters the ZO region, theAZ control unit 122 determines to start zooming out, and performs azoom-out operation corresponding to a predetermined zoom magnification.In addition, when the face tracking frame 800 d is included in the ZIregion, the AZ control unit 122 determines to start zooming in, andperforms a zoom-in operation corresponding to the predetermined zoommagnification to the zoom return position. If the ZO regions and the ZIregion are set in this manner, the movement of the object in thehorizontal direction is detected, and frame-out can be effectivelyprevented.

FIG. 8C shows a ZO region and a ZI region that are set when thedetection state of the shake detection unit 126 is a fixed state. Thefixed state is a state where the camera is fixed by a tripod or thelike, and there is no possibility of frame-out due to camera shake.Furthermore, if a zoom-in operation is performed when the object is notframed in near the center of the screen, there is a possibility offrame-out. In view of this, the ZO region is set around the entireperipheral portion of the screen, and the ZI region is set inside thezoom-in angle of view. Specifically, the ZO region is set in a form of arectangular band positioned at the ends of the screen in the verticaldirection and the horizontal direction, and the ZI region is set as arectangle positioned in a center portion of the screen. In this case,when the face tracking frame 800 e enters the ZO region, the AZ controlunit 122 determines to start a zoom-out operation, and performs azoom-out operation corresponding to a predetermined zoom magnification.Moreover, when the face tracking frame 800 f is included in the ZIregion, the AZ control unit 122 determines to start a zoom-in operation,and performs a zoom-in operation corresponding to the predetermined zoommagnification, to the zoom return position.

As described above, according to this embodiment, the ranges of the ZOregion and the ZI region are dynamically changed in accordance withchange in the attitude of the camera (vertical position/normal position)and change in shooting state (hand-held state/fixed state), and thusframe-out of the object can be effectively prevented while preventing amalfunction due to camera shake and the like. Note that the ZO regionand the ZI region may be changed in accordance with either the attitudeof the camera (vertical position/normal position) or the shooting state(hand-held state/fixed state), and only one of the ZO region and the ZIregion may be changed.

Next, size maintaining control for maintaining the ratio of an objectimage to the screen within a predetermined range during a zoom operationof this embodiment will be described with reference to FIGS. 9A to 10F.

In this embodiment, control (size maintaining control) is performed soas to maintain the size of an object image within the predeterminedrange from a reference size by automatically performing a zoom operationin the case where the size of the detected object image has changed bymore than a predetermined number of times the reference size. FIGS. 9Ato 9F show control for maintaining the size of an object that is aperson, and FIGS. 10A to 10F show control for maintaining the size of anobject other than a person.

First, a zoom operation related to control for maintaining the size ofan object that is a person will be described with reference to FIGS. 9Ato 9F. FIGS. 9A to 9C show a zoom operation automatically performed bythe camera in the case where the object (person) approaches the camera.This is a zoom-out operation for keeping the ratio of the object imageto the screen within the predetermined ratio. Note that in FIGS. 9A to9F, face tracking frames 900 a to 900 f are displayed so as to surroundthe face region as a characteristic region of the person that is theobject. Therefore, the size of the face tracking frame will be describedbelow as the size of the object.

FIG. 9A shows an angle of view when the object is designated inaccordance with the object designating method, which will be describedlater. When designating the object, the face tracking frame 900 a isdetermined based on the size of the designated object, and the size ofthe face tracking frame 900 a is stored as a reference object size(reference size) in the memory 118. FIG. 9B shows an angle of view whenthe object approaches the camera in the state where the zoommagnification is not changed from the state in FIG. 9A. For example, asize that is 150% of the size of the face tracking frame 900 a that isthe reference object size serves as a size when a zoom-out operation isstarted. When the relationship between object tracking frames (facetracking frames) becomes “face tracking frame 900 b>face tracking frame900 a×150%”, in other words, when the tracking frame changes by morethan a predetermined change amount relative to the reference size, theAZ control unit 122 determines to start a zoom-out operation.

FIG. 9C shows an angle of view zoomed out by a predetermined zoommagnification from the angle of view in FIG. 9B, and the face trackingframe 900 c. Here, the predetermined zoom magnification is set to 1/1.5in consideration of the change rate (150%) of the face tracking framesize when a zoom-out operation is started, relative to the referenceobject size. After that, in the case where the object further approachesthe camera, the object image can be continuously kept within thepredetermined ratio by further performing zooming out to the wide angleside. Therefore, the photographer is able to concentrate only on theoperation of the release switch.

On the other hand, FIGS. 9D to 9F show a zoom operation automaticallyperformed by the camera in the case where the person that is the objectmoves away from the camera. This is a zoom-in operation for keeping theratio of the object image to the screen within the predetermined range.FIG. 9D shows an angle of view when the object is designated inaccordance with the object designating method, which will be describedlater. The size of the face tracking frame 900 d when the object isdesignated is stored as a reference object size in the memory 118.

FIG. 9E shows an angle of view when the object moves away from thecamera in the state where the zoom magnification is not changed from thestate in FIG. 9D. For example, a size that is 50% of the size of theface tracking frame 900 d that is the reference object size serves as asize when a zoom-in operation is started. When the relationship betweenface tracking frames becomes “face tracking frame 900 e<face trackingframe 900 d×50%”, it is determined that the tracking frames have changedby more than a predetermined change amount relative to the referencesize. When this determination condition is satisfied, and the facetracking frame 900 e is included in the ZI region, the AZ control unit122 determines to start a zoom-in operation. Here, the ZI region is setwithin an angle of view zoomed in by a predetermined zoom magnificationrelative to the angle of view in FIG. 9E.

FIG. 9F shows the angle of view zoomed in by the predetermined zoommagnification from the angle of view in FIG. 9E, and the face trackingframe 900 f. Here, the predetermined zoom magnification is set to I/0.5in consideration of the change rate (50%) of the face tracking framesize when a zoom-in operation is started, relative to the referenceobject size.

Next, a zoom operation related to control for maintaining the size of anobject other than a person will be described with reference to FIGS. 10Ato 10F. If the object is a body (airplane) other than a face of aperson, a region of the same color is detected as the size of theobject, and thus a detection error is greater than a face region in thecase where the object is a face of a person. Accordingly, in thisembodiment, the accuracy of automatic zooming control is increased byusing the object distance acquired with the AF control unit 124 inaddition as the condition for determining to start a zoom operation. Therelationship between the object distance and the object size is aninversely proportional relationship. For example, in the case where anobject at a distance of 10 m from the digital camera 100 approaches halfthe distance to a distance of 5 m, the object size changes byapproximately twice. Therefore, the ratio of the object to the screencan be kept within the predetermined range by changing the zoommagnification by the inverse number of the change magnification of theobject size, namely, the change magnification of the object distance.Size maintaining control by a zoom operation of this embodiment will bedescribed specifically below with reference to FIGS. 10A to 10F. Notethat a zoom operation that is based on the object size is similar to thecase where the object is a face of a person, and thus descriptionthereof is omitted.

FIGS. 10A to 10C show a zoom operation automatically performed by thecamera in the case where an object (airplane) approaches the camera.This is a zoom-out operation for keeping the ratio of the object imageto the screen within a predetermined ratio. Note that in FIGS. 10A to10F, entity tracking frames 1000 a to 1000 f are displayed so as tosurround the body region as a characteristic region of the airplane thatis the object.

FIG. 10A shows an angle of view when the object is designated by theobject designating method, which will be described later. When theobject is designated, the distance to the designated object is acquiredby the AF control unit 124, and the acquired object distance isdetermined as a reference object distance (reference distance), and isstored in the memory 118. FIG. 10B shows an angle of view when theobject approaches the camera in the state where the zoom magnificationis not changed from the state in FIG. 10A. For example, a distance thatis ½ of the reference object distance serves as a distance when azoom-out operation is started. When the relationship of the objectdistance acquired by the AF control unit 124 becomes “objectdistance<reference distance/2”, the AZ control unit 122 determines tostart a zoom-out operation.

FIG. 10C shows an angle of view zoomed out by a predetermined zoommagnification from the angle of view in FIG. 10B, and the entitytracking frame 1000 c. Here, the predetermined zoom magnification is setto ½ in consideration of the change rate (½ times) when a zoom-outoperation is started, relative to the reference distance. After that, inthe case where the object further approaches the camera, the objectimage can be continuously kept within the predetermined ratio by furtherperforming zooming out to the wide angle side. Therefore, thephotographer is able to concentrate only on the operation of the releaseswitch.

On the other hand, FIGS. 10D to 10F show a zoom operation automaticallyperformed by the camera in the case where the airplane that is theobject moves away from the camera. This is a zoom-in operation forkeeping the ratio of the object image to the screen within thepredetermined range. FIG. 10D shows an angle of view when the object isdesignated in accordance with the object designating method, which willbe described later. The object distance when the object is designated isstored as a reference object distance in the memory 118.

FIG. 10E shows an angle of view when the object moves away from thecamera if the zoom magnification is not changed from the state in FIG.10D. For example, a distance that is twice the reference object distanceserves as a size when a zoom-in operation is started. When therelationship of the object distance acquired by the AF control unit 124becomes “object distance>reference distance×2”, it is determined thatthe distance has changed by more than a predetermined change amountrelative to the reference distance. When this determination condition issatisfied, and the entity tracking frame 1000 e is included in the ZIregion, the AZ control unit 122 determines to start a zoom-in operation.Here, the ZI region is set inside the angle of view zoomed in by apredetermined zoom magnification relative to the angle of view in FIG.10E.

FIG. 10F shows the angle of view zoomed in by the predetermined zoommagnification from the angle of view in FIG. 10E, and the entitytracking frame 1000 f. Here, the predetermined zoom magnification is setto 2 times in consideration of the change rate (2 times) of the objectdistance when a zoom-in operation is started, relative to the referencedistance.

As described above, according to this embodiment, in the case where theobject is not a face of a person, the size on the screen can bemaintained regarding one of or both the object distance and the objectby changing the zoom magnification in accordance with the changedmagnification.

Processing Procedure of Automatic Zooming Control Next, the processingprocedure of automatic zooming control of this embodiment will bedescribed with reference to the flowcharts in FIGS. 11 to 17B. Note thatthe processing in FIGS. 11 to 17B is realized by the system control unit114 executing a control program read out from the memory 118. Inaddition, the automatic zooming function of this embodiment is realizedby the system control unit 114 (the AZ control unit 122) controlling theconstituent elements based on control instructions, unless explicitlystated otherwise.

First, the processing procedure of overall automatic zooming controlwill be described with reference to FIG. 11.

In step S1100, the AZ control unit 122 waits for the automatic zoomingfunction to be set ON while monitoring the operation state of theautomatic zooming switch included in the operation unit 117, and whenthe automatic zooming function is set ON, the procedure advances to stepS1101. After that, the AZ control unit 122 sequentially executes objectdesignating processing (step S1101), reference size setting processing(step S1102), reference distance setting processing (step S1103) andautomatic zooming control (step S1104).

When the processing of steps S1101 to S1104 ends, the procedure advancesto step S1105. In step S1105, the AZ control unit 122 determines whetheror not the object that is being tracked has been changed. If the objectthat is being tracked has been changed, the procedure returns to stepS1101, and in the case where the object that is being tracked has beennot changed, the procedure advances to step S1106.

In step S1106, the AZ control unit 122 determines whether or not theautomatic zooming function has been set ON while monitoring theoperation state of the automatic zooming switch included in theoperation unit 117. If the automatic zooming function has not been setON, the processing ends, and if the automatic zooming function has beenset ON, the procedure returns to step S1104, where automatic zoomingcontrol is continued.

Object Specifying Processing Here, the object designating processing instep S1102 will be described with reference to FIG. 12. FIG. 12A showsprocessing for the photographer designating a desired object using thetouch panel included in the operation unit 117. In this embodiment, itis assumed that the photographer performs a designating operation bytouching an object image displayed on the display unit 109. In stepS1200, the AZ control unit 122 determines whether or not the touch panelhas been touched. If the touch panel has been touched, the procedureadvances to step S1201, and if the touch panel has not been touched,determination processing is repeated until the touch panel is touched.In step S1201, the AZ control unit 122 acquires information regardingthe position (touch position) that the photographer touched. In stepS1202, the AZ control unit 122 notifies the object detection unit 123 ofthe touch position, and the object detection unit 123 performs facedetection in the vicinity of the touch position. If a face is detectedin the vicinity of the touch position in step S1202, the AZ control unit122 determines that the main object is a person, and the procedureadvances to step S1203. In addition, if a face is not detected near thetouch position in step S1202, the AZ control unit 122 determines thatthe main object is a body other than a person, and the procedureadvances to step S1204.

In step S1203, the AZ control unit 122 stores, in the memory 118, faceinformation of the person that is to be subjected to automatic tracking.Such face information includes information such as the size of the faceand the face detection position when the object is designated and theattitude of the face. In addition, in a camera that has a faceauthentication function, identification information such as anauthentication ID is also stored in the memory 118.

In step S1204, the AZ control unit 122 stores, in the memory 118,characteristic colors in the vicinity of the touch position as colorinformation to be subjected to automatic tracking. Such colorinformation includes characteristic colors and the luminance thereofwhen the object is designated, the value of color difference, the sizeof regions of the same color, and the centroid position of the region ofthe same color. In addition, in a camera that has an entityauthentication function, identification information such as anauthentication ID is also stored in the memory 118. Hereinafter, faceinformation and color information are also referred to collectively asobject information (including the position and the size of the object).

In step S1205, the AZ control unit 122 displays an object trackingframe. Specifically, the AZ control unit 122 causes the display unit 109to display an object tracking frame (an entity tracking frame or a facetracking frame) whose size corresponds to the object size, centered onthe object detection position, and object designating processing ends.

In this manner, in the system control unit 114, the object detectionunit 123 detects an object in the display unit 109 at a positiondesignated by the photographer or in the vicinity of the designatedposition. The AZ control unit 122 then displays an object tracking frameon the display unit 109. According to the processing in FIG. 12A, thephotographer can designate an object that he or she wants to track,easily by an intuitive method. Note that the object designating methodis not limited to the above-described method. FIG. 12B shows processingif the photographer designates an object using a switch (an objectdesignating switch) other than the automatic zooming switch as anoperation member that constitutes the operation unit 117, as adesignating method other than that in FIG. 12A.

In step S1206, the AZ control unit 122 displays, in the vicinity of thecenter of the screen of the display unit 109, an object designatingframe serving as a guide for designating an object. The photographeradjusts the attitude of the camera using this object designating frameas a guide, such that an object image desired to be tracked is includedin the vicinity of the center of the screen.

In step S1207, the AZ control unit 122 determines whether or not theobject designating switch has been pressed. If the object designatingswitch has been pressed, the procedure advances to step S1208, and ifthe switch has not been pressed, determination processing is repeateduntil the switch is pressed.

In step S1208, the object detection unit 123 performs face detectionnear the center of the screen. If a face is detected, the AZ controlunit 122 determines that the main object is a person, and the procedureadvances to step S1209. On the other hand, if a face is not detected,the AZ control unit 122 determines that the main object is a body otherthan a person, and the procedure advances to step S1210.

In steps S1209 to S1211 in FIG. 12B, processing similar to theprocessing of steps S1203 to S1205 in FIG. 12A is performed.

In this manner, in the processing in FIG. 12B, the object detection unit123 detects an object at the center position of the screen or in thevicinity of the center position of the screen of the display unit 109.The AZ control unit 122 displays, on the display unit 109, an objecttracking frame indicating the position of the object. Even with a camerathat is not equipped with an operation device such as a touch panel, thephotographer can easily designate an object.

FIG. 12C shows processing for automatically selecting an object to betracked from faces detected when the automatic zooming switch includedin the operation unit 117 was pressed, as a method other than thatdescribed with reference to FIG. 12A.

In step S1212, the object detection unit 123 performs face detection onthe entire screen, and determines whether or not a face of a person hasbeen detected. If even a single face is detected in the entire screen,the object detection unit 123 determines that the main object is aperson, and the procedure advances to step S1213. In addition, if not aface is detected in the screen, the object detection unit 123 advancesthe procedure to step S1217.

If a face of a single person has been detected, the object detectionunit 123 selects the face as a main face in step S1213. If a pluralityof faces have been detected, the object detection unit 123 selects, fromthose faces, a main face that is an object to be tracked. For example,there is a method for preferentially selecting a face whose facedetection position is closer to the center position of the screen, as adetermination reference for selecting a main face. In addition, there isa method for selecting a face whose size is larger as a main face if aplurality of faces exist at positions that are similar distances fromthe center of the screen. Moreover, in the case of a camera that has aface authentication function, there is a method in which, if there is aface registered for authentication, the face is preferentially set as amain face.

In step S1214, similarly to step S1203, the AZ control unit 122 storesface information of the selected main face in the memory 118, and theprocedure advances to step S1215. In step S1215, the AZ control unit 122displays a face tracking frame on the screen similarly to step S1205. Instep S1216, the AZ control unit 122 performs face change determination.Specifically, if a plurality of faces have been detected, theautomatically selected main face is not necessarily a face intended bythe photographer. Therefore, the photographer can change the main face.At this time, when the photographer designates a desired face bypressing the operation unit 117 (the automatic zooming switch or thelike), processing for the face tracking frame changing the main face toanother face that has not been selected as a main face from detectedfaces is performed. If the main face is changed in step S1216, theprocedure returns to step S1214, where the AZ control unit 122 updatesface information to be stored in the memory 118, and in step S1215, theface tracking frame is changed to the size and the detection position ofthe newly selected main face.

On the other hand, if the procedure has advanced from step S1212 to stepS1217, the object detection unit 123 determines that the main object isa body other than a person, and the AZ control unit 122 stores, in thememory 118, a characteristic color in the vicinity of the center of thescreen as color information to be subjected to automatic tracking,similarly to step S1204. In step S1218, the AZ control unit 122 displaysan entity tracking frame on the screen similarly to step S1205, and endsthe processing.

In this manner, in the processing in FIG. 12C, the object detection unit123 of the system control unit 114 performs face detection on the entirescreen of the display unit 109. If a plurality of faces has beendetected, the AZ control unit 122 displays, on the display unit 109, anobject tracking frame indicating the position of a first face as anobject out of the faces. In addition, if the object has been changedfrom the first face to a second face, the AZ control unit 122 displaysan object tracking frame indicating the position of the second face onthe display unit 109. Accordingly, the photographer can easily designatea desired object by performing the operations a fewer number of times.

Note that regarding object designating processing, if the camera isprovided with both the touch panel and the object designating switchincluded in the operation unit 117, both the method in FIG. 12A and themethod in FIG. 12B may be applied at the same time. Moreover, objectdesignating processing to be applied may be changed in accordance withthe processing flow.

Reference Distance Setting Processing Next, reference distance settingprocessing of step S1103 will be described with reference to FIG. 13.

The AZ control unit 122 stores the object distance to the objectdesignated by performing object designating processing in step S1102 inFIG. 11, as a reference distance in the memory 118, and refers to thereference distance in automatic zooming control, which will be describedlater.

In step S1300, the AZ control unit 122 determines whether or not theobject designated in object designating processing is a face of aperson. If it is determined in step S1300 that the object is a face of aperson, the processing ends. If the object is a face of a person, thereliability of the detected size of the face region is relatively high,and thus the AZ control unit 122 executes automatic zooming controlbased only on the object size without referring to information regardingthe object distance. If it is determined in step S1300 that the objectis a body, the procedure advances to step S1301. If the object is a bodyother than a face of a person, the AZ control unit 122 uses informationregarding both the object size and the object distance of acharacteristic color region to execute automatic zooming control.

In step S1301, the AZ control unit 122 determines whether or not thefocus flag is ON, namely, the object is focused, with an AF controlstate in the continuous AF mode by the AF control unit 124 being thestopped state 502 in FIG. 5. If the object is not focused on, the AZcontrol unit 122 repeats determination processing until the object isfocused by AF control, and if it is determined that the object isfocused, the procedure advances to step S1302.

In step S1302, the AZ control unit 122 determines whether or not thefocus change rate is greater than or equal to a predetermined value. Asa method for calculating the focus change rate, the AF control unit 124acquires the object distance from the focus position in an in-focusstate and the focal length corresponding to the zoom position withreference to the focus cam table shown in FIG. 6A. Furthermore, inrelevant focal length data in the focus cam table shown in FIG. 6A, theinclination near the acquired object distance is calculated and is setas the focus change rate. If the calculated focus change rate is smallerthan the predetermined value, the reliability of the acquired objectdistance is low, and thus the AZ control unit 122 ends the processingwithout storing the object distance as a reference distance in thememory 118, and if the focus change rate is greater than or equal to thepredetermined value, the procedure advances to step S1303.

In step S1303, the AZ control unit 122 stores the acquired objectdistance as a reference distance in the memory 118, and ends theprocessing.

Automatic Zooming Control Next, the automatic zooming control of stepS1104 in FIG. 11 will be described with reference to FIG. 14.

In step S1400, the AZ control unit 122 determines whether or not theobject detection unit 123 has detected an object. If an object has notbeen detected, the processing ends, and if an object has been detected,the procedure advances to step S1401.

In step S1401, the AZ control unit 122 determines whether or not theobject detected in step S1400 is a face of a person. If it is determinedthat the object is a face of a person, the procedure advances to stepS1403, and if it is determined that the object is a body, the procedureadvances to step S1402.

In step S1402, the object is a body, and thus the AZ control unit 122determines the probability as a region of an object to be subjected toautomatic zooming control, based on the characteristic color of theobject detected by the object detection unit 123. The object detectionunit 123 calculates object likelihood that indicates the probability ofthe detected characteristic color as an object. The object likelihood iscalculated based on the difference between the characteristic color ofthe designated object stored in the memory 118 in steps S1204, S1210 andS1217 in FIG. 12A to 12C and the detected characteristic color, whetheror not a similar characteristic color exists in the image, colordistribution in the entire image (whether or not the object color andthe background color can be distinguished), and the like. If the objectlikelihood is smaller than a predetermined value, the AZ control unit122 ends the automatic zooming control in order to prevent a malfunctiondue to the reliability of the position and the size of the detectedobject being low. In this case, the display unit 109 may displaynotification that automatic zooming control cannot be executed. Forexample, if an icon displays that the automatic zooming function is ON,a method for graying out the color of the icon, a method for graying outthe color of the entity tracking frame, a method for displaying the text“object cannot be detected” or the like may be applied, for example. Inaddition, if the object likelihood is greater than or equal to thepredetermined value, the procedure advances to step S1403. Specifically,if it is determined that the object is a face of a person or a body forwhich the reliability of object detection is high, frame-out preventioncontrol is executed in step S1403.

Frame-Out Prevention Control Here, frame-out prevention control of stepS1403 will be described with reference to FIG. 15. As described withreference to FIGS. 7A to 8C, frame-out prevention control is executed inorder to prevent an object from being framed out of the screen.

In step S1500, the AZ control unit 122 determines whether or not theobject tracking frame related to the object that is tracked has entereda ZO region. The ZO region is equivalent to the ZO region described withreference to FIG. 7A to 8C. If it is determined in step S1500 that theobject tracking frame has entered the ZO region, namely, if it is highlypossible that the object will be framed out, the procedure advances tostep S1503, where a zoom-out operation is started. The zoom-outoperation here is equivalent to the zoom-out operation by frame-outprevention control. After the zoom-out operation, frame-out preventioncontrol ends. On the other hand, if it is determined in step S1500 thatthe object tracking frame has not entered the ZO region, namely, if thecamera captures the object near the center of the screen, the procedureadvances to step S1501.

In step S1501, the AZ control unit 122 determines whether or not theimmediately preceding zoom operation is a zoom-out operation due to theobject tracking frame entering the ZO region in step S1500, namely, azoom-out operation by frame-out prevention control. If it is determinedin step S1501 that a zoom-out operation by frame-out prevention controlhas been performed, the procedure advances to step S1502, and if it isdetermined in step S1501 that a zoom-out operation by frame-outprevention control has not been performed, the processing ends.

In step S1502, the AZ control unit 122 determines whether or not theobject tracking frame related to the object to be tracked fits within(is contained in) a ZI region. The ZI region is equivalent to the ZIregion described with reference to FIGS. 7B to 8C. If it is determinedin step S1502 that the object tracking frame does not fit within the ZIregion, the processing ends, and if it is determined that the objecttracking frame fits within the ZI region, the procedure advances to stepS1504. Accordingly, if the camera captures the object near the center ofthe screen and at the zoom return position such that the object size iswithin the angle of view, a zoom-in operation is started in step S1504.The zoom-in operation here is equivalent to the zoom-in operation byframe-out prevention control.

In step S1505, it is assumed that frame-out prevention control has beenexecuted due to the object having moved, and thus the AZ control unit122 updates the reference distance by performing reference distancesetting processing in FIG. 13, and ends the processing.

Returning to the description with reference to FIG. 14, when theframe-out prevention control of step S1403 ends, the procedure advancesto step S1404. In step S1404, the AZ control unit 122 determines whetheror not the immediately preceding zoom operation is a zoom-out operationcaused by the object tracking frame entering the ZO region in step S1500in FIG. 15, namely, a zoom-out operation by frame-out preventioncontrol. If it is determined in step S1404 that a zoom-out operation byframe-out prevention control has been performed, the AZ control unit 122ends the processing, and if it is determined that a zoom-out operationby frame-out prevention control has not been performed, the procedureadvances to step S1405, where size maintaining control is performed. Inthis embodiment, in order to successfully perform both frame-outprevention control and size maintaining control in the automatic zoomingfunction, first, the object is placed in near the center of the screenby performing frame-out prevention control to enable size maintainingcontrol to be executed. Therefore, after the zoom-out operation byframe-out prevention control, size maintaining control for maintainingthe object size constant, which will be described later, is notexecuted. In other words, if frame-out prevention control has beenexecuted, execution of size maintaining control is restricted until thezoom-in operation by frame-out prevention control is complete.

Size Maintaining Control of First Embodiment

Next, the size maintaining control of step S1405 in FIG. 14 will bedescribed with reference to FIGS. 16 to 17B. As described with referenceto FIGS. 9A to 10F, size maintaining control is performed in order tomaintain the size of the object image within a predetermined range fromthe reference size. FIG. 16 shows size maintaining control of the firstembodiment. In the first embodiment, if the object is a body other thana face of a person and the reliability of the object distance is high,automatic zooming control is executed using only the object distance(first information), and if the reliability of the object distance islow, automatic zooming control is executed using only the object size(second information). The processing of steps S1600 to S1605 isprocessing for setting a condition for determining to start a zoomoperation.

In step S1600, the AZ control unit 122 determines whether or not theobject detected in step S1400 is a face of a person. If it is determinedthat the object is a face of a person, the procedure advances to stepS1601, and if it is determined that the object is a body other than aface of a person, the procedure advances to step S1602.

In step S1601, the reliability of the object size is relatively high,and thus the AZ control unit 122 sets the change magnification of theobject size used for determination for starting a zoom operation to 1.4times that is smaller than the change magnification of a body other thana person, thereby ensuring high responsiveness to change in object size.

In step S1602, the AZ control unit 122 determines whether or not the AFcontrol state in the continuous AF mode by the AF control unit 124 isthe stopped state 502 in FIG. 5, and the focus flag is ON, namely, theobject is focused. If it is determined in step S1602 that the object isfocused, the procedure advances to step S1603, and if it is determinedthat the object is not focused, the procedure advances to step S1604.

In step S1603, the AZ control unit 122 determines whether or not thefocus change rate has become greater than or equal to a predeterminedvalue in the reference distance setting processing in FIG. 13 and areference distance has been set. If it is determined in step S1603 thata reference distance has not been set, the procedure advances to stepS1604, and if it is determined that a reference distance has been set,the procedure advances to step S1605. Accordingly, if it is determinedin step S1602 that the object is not focused, or if it is determined instep S1603 that a reference distance has not been set, the reliabilityof the acquired object distance is low, and thus automatic zoomingcontrol is executed using only the object size information.

In step S1604, the AZ control unit 122 sets the change magnification ofthe object size used for determination for starting a zoom operation totwo times, which is greater than the change magnification of a person.Accordingly, if it is determined in step S1602 that the object isfocused, and it is determined in step S1603 that a reference distancehas been set, automatic zooming control is executed using only theobject distance information. In step S1605, the AZ control unit 122 setsthe change magnification of the object distance used for determinationfor starting a zoom operation to 1.6 times.

In the processing of steps S1606 to S1612, determination for starting azoom operation is performed based on the change magnification of theobject size and the object distance, and a zoom operation is executed.If it is determined in step S1600 that the object is a face of a person,if it is determined in steps S1602 and S1603 that the object is a bodyother than a face of a person, and the reliability of the acquiredobject distance is low, the procedure advances to step S1606. In stepS1606, the AZ control unit 122 compares the detected size of the objectdetected in step S1400 in FIG. 14 to the size obtained by multiplyingthe reference size by the magnification N set in step S1601 or S1604 (Ntimes). If the detected size of the object is greater than N times thereference size, in other words, if the rate of the object image to thescreen exceeds a predetermined value, the procedure advances to stepS1611.

In step S1611, the AZ control unit 122 starts a zoom-out operation. Thiszoom-out operation is equivalent to the zoom-out operation for sizemaintaining control. On the other hand, if in step S1606, the detectedsize of the object is smaller than or equal to N times the referencesize, the procedure advances to step S1607.

In step S1607, the AZ control unit 122 compares the detected size of theobject to the reference size. If the detected size of the object issmaller than the size obtained by dividing the reference size by themagnification N set in step S1601 or S1604 (1/N times), in other words,if the ratio of the object image to the screen is smaller than thepredetermined value, the procedure advances to step S1610. On the otherhand, if the detected size of the object is greater than or equal to 1/Ntimes the reference size, automatic zooming control ends.

In step S1610, the AZ control unit 122 determines whether or not theobject tracking frame related to the object to be tracked fits within(is included in) the ZI region. This is for preventing the object frombeing framed out due to a zoom-in operation, if the object is in theperiphery of the screen. This ZI region is equivalent to the ZI regiondescribed with reference to FIGS. 9E and 10E. If it is determined instep S1610 that the object tracking frame does not fit within the ZIregion, automatic zooming control ends, and if it is determined that theobject tracking frame fits within the ZI region, the procedure advancesto step S1612.

In step S1612, the AZ control unit 122 starts a zoom-in operation. Thiszoom-in operation is equivalent to the zoom-in operation for sizemaintaining control. In this manner, in this embodiment, in order toalso prevent frame-out of the object in a zoom-in operation for sizemaintaining control, the zoom-in operation is started after the objectimage fits within the ZI region. After the zoom-in operation, theprocedure advances to step S1613.

In step S1613, the AZ control unit 122 updates the reference distance byperforming reference distance setting processing in FIG. 13, and endsthe processing.

On the other hand, if it is determined in the determination in stepsS1602 and S1603 that the object is a body other than a face of a personand the reliability of the acquired object distance is high, theprocedure advances to step S1608. In step S1608, the AZ control unit 122compares the acquired object distance to the distance obtained bydividing the reference distance by the magnification set in step S1605(1/M times). If the object distance acquired by the AF control unit 124is smaller than 1/M times the reference distance, in other words, if theobject has approached by more than a predetermined distance, theprocedure advances to step S1611.

In step S1611, the AZ control unit 122 starts a zoom-out operation. Onthe other hand, if in step S1608, the acquired object distance isgreater than or equal to 1/M times the reference distance, the procedureadvances to step S1609. In step S1609, the AZ control unit 122 comparesthe acquired object distance with the reference distance. If theacquired object distance is greater than the size obtained bymultiplying the reference distance by the magnification set in stepS1605 (M times), in other words, if the object has receded by more thana predetermined distance, the procedure advances to step S1610. On theother hand, if the acquired object distance is less than or equal to Mtimes the reference distance, automatic zooming control ends.

Size Maintaining Control of Second Embodiment Next, size maintainingcontrol of a second embodiment will be described with reference to FIGS.17A and 17B. In the second embodiment, if the object is a body otherthan a face of a person and the reliability of the object distance ishigh, automatic zooming control is executed using both the objectdistance (first information) and the object size (second information),and if the reliability of the object distance is low, automatic zoomingcontrol is executed using only the object size (second information).Processing of determination for starting a zoom operation in the casewhere the object is a body other than a face of a person and thereliability of the acquired object distance is high is different betweenthe first embodiment and the second embodiment. Description will begiven below focused on the differences.

The processing of steps S1700 to S1704 is similar to the processing ofsteps S1600 to S1604 in FIG. 16. If it is determined in steps S1700,S1702 and S1703 that the object is a body other than a face of a personand the reliability of the acquired object distance is high, theprocedure advances to step S1705. In step S1705, the AZ control unit 122compares a value obtained by dividing the reference size by the detectedsize, which is the change magnification of the object size, to a valueobtained by dividing the acquired distance by the reference distance,which is the change magnification of the object distance, and determineswhether or not the difference between the change magnification of theobject size and the change magnification of the object distance issmaller than a predetermined value. Accordingly, if change in objectsize and change in object distance are based on almost the same changemagnification, the reliability of the detected size and the acquireddistance of the object can be estimated to be high. In view of this, thecondition for determining to start a zoom operation is changed based ona result of comparing the change magnification of the object size to thechange magnification of the object distance. If it is determined in stepS1705 that the difference between the change magnification of the objectsize and the change magnification of the object distance is smaller thana predetermined value, the procedure advances to step S1706. In stepsS1706 and S1707, the difference between the change magnification of theobject size and the change magnification of the object distance issmall, and thus the AZ control unit 122 sets the change magnification ofthe object distance and the object size, which serves as the conditionfor determining to start a zoom operation, to 1.6 times. On the otherhand, if it is determined in step S1705 that the difference between thechange magnification of the object size and the change magnification ofthe object distance is greater than or equal to the predetermined value,the procedure advances to step S1708. In steps S1708 and S1709, thedifference between the change magnification of the object size and thechange magnification of the object distance is large, and thus the AZcontrol unit 122 sets the change magnification for the object distanceand the object size, which serves as the condition for determining tostart a zoom operation, to 2 times.

In step S1713, the AZ control unit 122 compares the acquired objectdistance to a distance obtained by dividing the reference distance bythe magnification set in step S1706 or S1708 (1/M times). If theacquired object distance is smaller than 1/M times the referencedistance, in other words, if it is determined that the object hasapproached by a predetermined distance, the procedure advances to stepS1715. In step S1715, the AZ control unit 122 starts a zoom-outoperation. On the other hand, if the acquired object distance is greaterthan or equal to 1/M times the reference distance, the procedureadvances to step S1714.

In step S1714, the AZ control unit 122 compares the acquired objectdistance with a distance obtained by multiplying the reference distanceby the magnification set in step S1706 or S1708 (M times). If theacquired object distance is greater than M times the reference distance,in other words, if it is determined that the object has receded by morethan the predetermined distance, the procedure advances to step S1712.On the other hand, if the acquired object distance is smaller than orequal to M times the reference distance, the procedure advances to stepS1710. In size maintaining control of the second embodiment, if movementof the object has not been determined using the object distance,determination of movement of the object is continuously performed againusing the object size. In steps S1710 to S1717, similarly to theprocessing of steps S1606 to S1613 in FIG. 16, object size changedetermination is performed using the magnification set in step S1707 orS1709.

Zoom Operation

Next, the zoom-out operations or the zoom-in operations in steps S1503and S1504 in FIG. 15, steps S1611 and S1612 in FIG. 16, and steps S1715and S1716 in FIG. 17A will be described with reference to FIG. 18.

In step S1800, the AZ control unit 122 acquires a zoom change amount (achange amount of zoom magnification) from the memory 118. In the case ofa zoom-out operation by frame-out prevention control, the zoom-outchange amount is set in accordance with detected object information.Specifically, in the zoom-out operation by frame-out prevention control(step S1503 in FIG. 15), the smaller the size of the object is, thesmaller the zoom-out change amount is set. Accordingly, it is possibleto avoid a state in which the object cannot be detected due to the sizeof the object having been reduced excessively by the zoom-out operation.Note that a configuration may be adopted in which, in consideration ofthe minimum size with which the object can be detected, a zoom-outoperation is not performed if the size of the object is smaller than apredetermined size. In addition, in a zoom-in operation by frame-outprevention control, the zoom magnification before the zoom-out operationis started is stored in the memory 118, and the zoom-in change amount isset such that a zoom magnification that is the same as that beforestarting the zoom-out operation is obtained.

Moreover, in a zoom-out operation based on the object size in sizemaintaining control (step S1611 in FIG. 16, step S1715 in FIG. 17A), azoom-out change amount (1/N times) corresponding to predetermined Ntimes used for determination in step S1606 or S1710 is set. Accordingly,even if an object cannot be detected, a minimum zoom-out operation canbe performed until the size of the object reaches a reference objectsize. Similarly, also in the case of a zoom-in operation based on theobject size in size maintaining control (step S1612 in FIG. 16, stepS1716 in FIG. 17A), a zoom-in change amount (N times) corresponding topredetermined 1/N times used for determination in step S1607 or S1711 isset. Furthermore, in a zoom-out operation based on the object distancein size maintaining control (step S1608 in FIG. 16, step S1713 in FIG.17B), a zoom-out change amount (1/M times) corresponding topredetermined 1/M times used for determination in step S1608 or S1713 isset. Accordingly, even if the object size is not determined, a zoom-outoperation can be performed such that the size of the object becomessubstantially the same. Similarly, also in the case of a zoom-inoperation based on the object distance in size maintaining control (stepS1609 in FIG. 16, step S1714 in FIG. 17B), a zoom-in change amount (Mtimes) corresponding to predetermined M times used for determination instep S1609 or S1714 is set.

In step S1801, the AZ control unit 122 sets, for the CZ control unit 119or the electronic zooming control unit 120, the zoom change amountacquired in step S1800, and starts resizing processing.

In step S1802, the AZ control unit 122 determines whether a zoomoperation of either frame-out prevention control or size maintainingcontrol is being performed. If the current zoom operation is a zoomoperation by frame-out prevention control (step S1503 and S1504 in FIG.15), the procedure advances to step S1806. In addition, if the currentzoom operation is a zoom operation for size maintaining control (stepsS1611 and S1612 in FIG. 16, steps S1715 and S1716 in FIG. 17A), theprocedure advances to step S1803.

In step S1803, the AZ control unit 122 determines whether the sizemaintaining control is based on the object size or the object distance.In the case of size maintaining control that is based on the objectsize, the procedure advances to step S1804, and in the case of sizemaintaining control that is based on the object distance, the procedureadvances to step S1806. In step S1804, the AZ control unit 122determines whether or not an object has been detected by the objectdetection unit 123. If an object has been detected, the procedureadvances to step S1805, and if an object has not been detected, theprocedure advances to step S1806.

In step S1805, the AZ control unit 122 compares a reference object sizeindicated by reference object information to the size of the objectdetected in step S1804. If the size of object detected in step S1804 andthe reference object size do not fit within a range of a predeterminedratio (within a predetermined change amount), the procedure returns tostep S1802, where zoom operation determination is continued. If the sizeof the object detected in step S1804 and the reference object size arebrought within the range of a predetermined ratio by a zoom operation,the procedure advances to step S1807. In step S1807, the AZ control unit122 stops the zoom operation, and ends the processing.

If it is determined in step S1802 that a zoom operation by frame-outprevention control is being performed, if it is determined in step S1803that a zoom operation based on the object distance in size maintainingcontrol is being performed, or if an object is not detected during azoom operation that is based on the object size in size maintainingcontrol in step S1804, the procedure advances to step S1806. In stepS1806, the AZ control unit 122 determines, based on the zoom changeamount acquired in step S1800, whether or not resizing processingequivalent to a predetermined zoom change amount for each zoom operationhas been performed. If resizing processing by the predetermined zoomchange amount has not been performed, the procedure returns to stepS1802, where the processing is continued. In addition, if resizingprocessing by the predetermined zoom change amount has been performed,the procedure advances to step S1807, where the AZ control unit 122stops the zoom operation and ends the processing.

As described above, according to this embodiment, automatic zoomingcontrol is executed based on the position and the size of an objectcalculated by the object detection unit 123 and the distance to theobject acquired by the AF control unit 124. In addition, automaticzooming is prevented from malfunctioning by changing zooming controlbased on the reliability of the position and the size of the object andthe reliability of the distance to the object. Accordingly, even in thecase of an object other than a person, it is possible to performappropriate automatic zooming control. Note that, in this embodiment,the object distance is acquired using the zoom position, the focusposition and the focus cam table, but a method for obtaining the objectdistance is not limited thereto, and for example, the object distancemay be obtained by measuring the distance to the object using a laser.Moreover, if information regarding the object distance can be obtained,it is not necessary to obtain the object distance itself. For example,instead of providing a threshold for the object distance, a thresholdmay be provided for a focus position for each zoom position.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2016-054469, filed Mar. 17, 2016 which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. A zooming control apparatus comprising: at leastone processor and a memory holding a program which makes the processorfunction as: an object detection unit configured to detect an objectfrom an image acquired through an imaging optical system; a firstacquisition unit configured to acquire information regarding a distanceto the object; and a zooming control unit configured to perform one of afirst zooming control for automatically changing a zoom magnificationaccording to first information that includes information regarding asize of the object detected by the object detection unit and a secondzooming control, that is different from the first zooming control, forautomatically changing a zoom magnification according to secondinformation regarding the distance to the object acquired by the firstacquisition unit, wherein the zooming control unit determines whether toperform the first zooming control or the second zooming control inaccordance with a focus state of the imaging optical system.
 2. Theapparatus according to claim 1, wherein when the imaging optical systemis focused on the object, the zooming control unit performs the secondzooming control.
 3. The apparatus according to claim 2, wherein when theimaging optical system is not focused on the object, the zooming controlunit performs the first zooming control.
 4. The apparatus according toclaim 1, wherein the information regarding a distance to the object isnot used in the first zooming control.
 5. The apparatus according toclaim 1, wherein the size of the object is a size of the object on theimage.
 6. A zooming control apparatus comprising: at least one processorand a memory holding a program which makes the processor function as: anobject detection unit configured to detect an object from an imageacquired through an imaging optical system; a first acquisition unitconfigured to acquire information regarding a distance to the object;and a zooming control unit configured to perform one of a first zoomingcontrol for automatically changing a zoom magnification according tofirst information that includes information regarding a size of theobject detected by the object detection unit and a second zoomingcontrol, that is different from the first zooming control, forautomatically changing a zoom magnification according to secondinformation regarding the distance to the object acquired by the firstacquisition unit, wherein the zooming control unit determines whether toperform the first zooming control or the second zooming control based ona relationship between a position of a focus lens included in theimaging optical system and a focal length of the imaging optical system.7. The apparatus according to claim 6, wherein the first acquisitionunit acquires the information regarding a distance to the object basedon the position of the focus lens and the focal length of the imagingoptical system, and the zooming control unit performs the second zoomingcontrol when an amount of change in the position of the focus lens perthe distance to the object is greater than a predetermined threshold. 8.The apparatus according to claim 6, wherein the first acquisition unitacquires the information regarding the distance to the object based onthe position of the focus lens and the focal length of the imagingoptical system, and the zooming control unit performs the first zoomingcontrol when the distance to the object is larger than a predeterminedthreshold.
 9. The apparatus according to claim 8, wherein thepredetermined threshold is set in accordance with the focal length. 10.The apparatus according to claim 9, wherein the predetermined thresholdwhen the focal length is a first value is larger than the predeterminedthreshold when the focal length is a second value, the first value issmaller than the second value.
 11. The apparatus according to claim 6,wherein the size of the object is a size of the object on the image. 12.A control method of a zooming control apparatus comprising: detecting anobject from an image acquired through an imaging optical system;acquiring information regarding a distance to the object; determiningwhether to perform one of a first zooming control and a second zoomingcontrol that is different from the first zooming control in accordancewith a focus state of the imaging optical system; and performing azooming control for automatically changing a zoom magnification based onthe determination, wherein the first zooming control automaticallychanges a zoom magnification according to first information thatincludes information regarding a size of the detected object, and thesecond zooming control automatically changes a zoom magnificationaccording to second information regarding the distance to the object.13. A control method of a zooming control apparatus comprising:detecting an object from an image acquired through an imaging opticalsystem; acquiring information regarding a distance to the object;determining whether to perform one of a first zooming control and asecond zooming control that is different from the first zooming controlbased on a relationship between a position of a focus lens included inthe imaging optical system and a focal length of the imaging opticalsystem; and performing a zooming control for automatically changing azoom magnification based on the determination, wherein the first zoomingcontrol automatically changes a zoom magnification according to firstinformation that includes information regarding a size of the detectedobject and the second zooming control automatically changes a zoommagnification according to second information regarding the distance tothe object.